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ABSTRACT 



This report presents findings of a study of instruments used 
in the identification of gifted and talented students. The study first 
examined the published literature, both standardized and locally developed 
identification instruments and procedures, and strategies used to identify 
underserved populations. These data were catalogued in the National 
Repository computer database. The study then reviewed standardized 
instruments using the "Scale for the Evaluation of Gifted Identification 
Instruments" for each construct of giftedness named by schools and these 
reviews were entered into the database. A review of identification procedures 
led to the compilation of standards for identification. Finally, data were 
collected on three locally developed instruments with potential for providing 
unique types of data for screening and identifying talent : the "Diet Cola 
Test" (better for program evaluation than identification) ; the "Peer Referral 
Form" (high reliability and useful with Hispanic populations) ; and the 
"Teacher Search List" (reliable in assessing middle school students) . Among 
seven appendices are the database order form, sample database output, a 
listing of instruments reviewed, the "Scale for the Evaluation of Gifted 
Identification Instruments" and the "Peer Referral Form." Contains 102 
references. (DB) 
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ABSTRACT 



The study of instruments used in the identification of gifted and talented students had 
multiple facets. First, published literature, standardized and locally developed identification 
instruments and procedures, and strategies used to identify underserved populations were 
collected and catalogued in a computer database. Then standardized instruments were 
reviewed using the Scale for the Evaluation of Gifted Identification Instruments for each 
construct of giftedness that schools named as an area in which they identified gifted 
students. These reviews were also entered in the database. The review of identification 
procedures led to the compilation of standards for identification. In addition, descriptions 
of school systems from this sample which exhibited innovative, exemplary practice and a 
selected group of innovative Javits projects were described in a monograph entitled Contexts 
for Promise: Noteworthy Practices and Innovations in the Identification of Gifted 
Students. 

Finally, data were collected on three locally developed instruments with potential for 
providing unique types of data for screening and identifying talent. The first instrument, the 
Diet Cola Test, was found to be reliable for group assessment purposes and useful as a 
program evaluation tool rather than as an identification instrument. The second instrument, 
a Peer Referral Form, was found to have high reliability and exhibited validity as 
recommended for a nomination form in the screening of Hispanic populations. Finally, the 
Teacher Search List was found to be reliably used by teachers in assessing middle school 
students. 



* This report is first of two technical reports which summarize the research project entitled "Investigations 
Into Instruments Used in the Identification of Gifted Students and the Evaluation of Gifted Programs." 
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EXECUTIVE SUMMARY 



The goals of the Identification and Evaluation Project conducted by The National 
Research Center on the Gifted and Talented (NRC/GT) at the University of Virginia were 
(a) to identify current practices in identifying gifted students and in evaluating gifted 
programs; (b) to collect relevant data on assessment instruments; (c) to evaluate those 
instruments using standards established by the measurement field; and (d) to identify 
pro mi sing practices in identification and evaluation. The first stage of the project was the 
establishment of a National Repository for Instruments and Strategies Used in the 
Identification and Evaluation of Gifted Programs. The second phase involved reviewing 
available data, including reliability and validity data, on identification and evaluation 
instruments in the Repository, and rating the instruments on their appropriateness for 
specific purposes. During the third phase of the project we investigated the effectiveness of 
pro mi sing non-published identification instruments. Studies of evaluation designs and the 
impact of evaluation practices on the utilization of evaluation findings were conducted 
concurrently and are described in a separate report. 



Overview of the Total Project 

The results of the project are presented in two separate reports. This technical report 
includes only research relating to identification. The evaluation collections and studies are 
reported in Instruments and Evaluation Designs Used in Gifted Programs. In a separate 
publication, practices for identifying students from at-risk populations were identified from 
entries in our data bank and the model projects funded through the Jacob K. Javits Gifted 
and Talented Students Education Act program. Descriptions of these practices were 
compiled into a monograph, Contexts for Promise: Noteworthy Practices in the 
Identification of Gifted Students (Callahan, Tomlinson, & Pizzat, n.d.). 

The initial focus of our investigation emphasized collecting and evaluating extant 
identification and evaluation literature, instruments, systems, and designs. The major 
research questions of the identification aspect of the study included: What are the most 
commonly used instruments in identifying gifted and talented students? What instruments 
are used for identifying gifted and talented students according to specific definitions and 
conceptions of giftedness? What is the reliability and validity evidence for these 
instruments, and is that evidence sufficient to justify their use with given definitions of 
giftedness and for identifying underserved populations? 



* This report is first of two technical reports which summarize the research project entitled "Investigations 
Into Instruments Used in the Identification of Gifted Students and the Evaluation of Gifted Programs," 



Similar questions were posed regarding evaluation instruments and designs: What 
instruments are die most commonly used in the evaluation of gifted students and programs? 
What are the reliability and validity of these instruments in assessing goals and objectives 
common to gifted programs? What instruments (especially non-traditional and product- 
oriented instruments) are used to evaluate programs for the gifted and talented? Which 
evaluation designs or which characteristics of evaluation designs yield useful evidence in 
program development and modification? 

During the second stage of this investigation, three non-published instruments 
potentially useful in identifying underserved gifted were selected for further investigation of 
their psychometric properties. The major research questions in this stage of the study were: 
What are the reliability and validity of each of these instruments? How effective are these 
instruments in identifying underserved populations of gifted students? In each case, we 
investigated the effectiveness of instruments relative to particular definitions of giftedness or 
the particular stated outcome goals of gifted programs. 

In preparing the monograph on promising practices (see Callahan, Tomlinson, & 
Pizzat, n.d.), the following questions were used as guides: Does this school system offer a 
set of innovative practices with documented evidence of effectiveness for identifying the 
underserved gifted? Do these systems used in identifying typically underserved gifted and 
talented students result in the identification of students who have special talents and needs? 



The Investigation of Identification Instruments 

The first step in the present study was to gather as many instruments and 
identification strategies as possible. The process was structured to gather information on 
both standardized and locally developed instruments and to identify state and local 
procedures for identification. Special efforts were made to identify instruments and 
strategies used in the identification of minority, economically disadvantaged, non-English 
speaking, and handicapped gifted students and in evaluating programs for these students. 

Database searches were conducted across Educational Resources Information 
Center (ERIC), PsycLIT (the computerized version of Psychological Index), Dissertation 
Abstracts International, and VIRGO (the University of Virginia computerized card 
catalogue system). Search terms included gifted, ratings, scales, reliability, validity, tests, 
measurements, identification, evaluation, and utilization, and these terms were used singly or 
in combination as appropriate. Each search yielded a list of potential resources which were 
reviewed for information on the state of the art in identification, information on use of 
particular instruments or strategies for identification, and information on reliability and 
validity. 

Information from the resources listed above was compiled into seven databases 
within the National Repository. The computer databases cover three categories of 
information: bibliographic entries, standardized instrument reviews and use, and locally 
developed materials. The bibliographic databases contain abstracts of published reviews of 
standardized instruments, abstracts of articles about the use of standardized instruments in 
identification, abstracts of articles about particular issues in identification (e.g., underserved 
populations). The standardized instrument databases include listings of the ways in which 
published instruments are used and reviews of the instruments on NRC/GT developed 
scales. The local instrument databases include listings of a collection of identification 
instruments developed and used at the local school level, but not published. 
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The staff of the project gathered all available data from the printed literature and 
from the survey responses on the reliability, validity, examinee appropriateness, norms, 
usability, teaching feedback, and ethical propriety of the instruments which were identified 
by school systems or the literature review. These technical data have been used to rate each 
instrument by means of a model rating scale developed by project staff. These ratings were 
also entered into the database. The project staff also reviewed all entries drawn from ERIC, 
the journal articles, and other sources on tests published between 1980 and 1990. These 
data have been condensed and entered into the databases. 

Reliability studies were completed on a peer nomination form ( Peer Referral Form), 
a test for identifying specific academic talent in science (Diet Cola Test), and two teacher 
rating scales ( Teacher Search List). All these instruments were found to be sufficiently 
reliable for group use, though one rating scale was found reliable only with upper 
elementary and intermediate level students. The science scale was not sufficiently reliable 
for individual assessments. Preliminary validity evidence suggests that the science scale is 
more valid for program outcome assessment, and that the peer assessment tool has potential 
for use in identifying Hispanic students. Neither instrument discriminated among different 
cultural groups. 
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CHAPTER 1: Introduction to the National Repository 



The goals of the Identification and Evaluation Project conducted by The National 
Research Center on the Gifted and Talented (NRC/GT) at the University of Virginia were 
(a) to identify current practices in identifying gifted students and in evaluating gifted 
programs, (b) to collect relevant data on assessment instruments, (c) to evaluate those 
instruments using standards established by the measurement field, and (d) to identify 
promising practices in identification and evaluation. The first stage of the project was the 
establishment of a National Repository for Instruments and Strategies Used in the 
Identification and Evaluation of Gifted Students Programs. The second phase involved 
reviewing available data, including reliability and validity data, on identification and 
evaluation instruments in the Repository and rating the instruments on their appropriateness 
for specific purposes. During the third phase we investigated the effectiveness of promising 
non-published identification instruments. Studies of evaluation designs and the impact of 
evaluation practices on the utilization of evaluation findings were conducted concurrently 
and are described in separate publication, Instruments and Evaluation Designs Used in 
Gifted Programs. Finally, promising innovative practices for identifying students from at- 
risk populations were identified from entries in our data bank and the model projects funded 
through the Jacob K. Javits Gifted and Talented Education Act program. Descriptions of 
these practices were compiled into a separate monograph, Contexts for Promise: 

Noteworthy Practices in the Identification of Gifted Students (Callahan, Tomlinson, & 
Pizzat, n.d.). 

The initial focus of our investigation emphasized collecting and evaluating extant 
identification and evaluation literature, instruments, systems, and designs. The major 
research questions posed for the identification aspect of the study included: What are the 
most commonly used instruments in identifying gifted and talented students? What 
instruments are used for identifying gifted and talented students according to specific 
definitions and conceptions of giftedness? What evidence is there of the reliability and 
validity of these instruments, and is that evidence sufficient to justify their use with given 
definitions of giftedness and for identifying underserved populations? 

Similar questions were posed regarding evaluation instruments and designs: What 
instruments are most commonly used in the evaluation of gifted students and programs? 
What are the reliability and validity of these instruments in assessing goals and objectives 
common to gifted programs? What instruments (especially non-traditional and product- 
oriented instruments) are used to evaluate programs for the gifted and talented? Which 
evaluation designs or which characteristics of evaluation designs yield useful evidence in 
program development and modification? 
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During the second stage of this investigation, three non-published specific 
instruments potentially useful in identifying underserved gifted were selected for further 
investigation of their psychometric properties. The major research questions in this stage of 
the study were: What are the reliability and validity of each of these instruments? How 
effective are these instruments in identifying underserved populations of gifted students? In 
each case, we investigated the effectiveness of instruments relative to particular definitions of 
giftedness or the particular stated outcome goals of gifted programs. 

In preparing the monograph on promising practices, we sought to identify school 
systems using identification procedures with documented evidence of effectiveness for 
identifying the underserved gifted. Procedures used to identify typically underserved gifted 
and talented students which resulted in the identification of students with special talents and 
needs and with a match between identification and programming were included in the 
monograph. 



Historical Perspective of the Identification Project 

A previous collection of identification instruments completed by the Educational 
Improvement Center (EIC) was limited only to identification and classification of 
instruments and did not include reviews of the instruments according to the gifted construct 
assumed by users of the instruments (Alvino, McDonnel, & Richert, 1981). General lists of 
instruments presented in most textbooks are most seriously deficient in their lack of 
evaluation of the reliability and validity of the instruments for specific gifted constructs, and 
many instruments with face validity have been included. Lists and evaluations of tests such 
as those provided in the Mental Measurements Yearbook, Tests in Print, and Tests fail to 
serve the purposes described above for several reasons. First, such reviews are directed at 
use by general audiences and for purposes other than the identification of the gifted or 
gifted program evaluation. They often fail to address the appropriateness of these 
instruments for the specialized purposes of identifying gifted and talented students or for 
evaluating the goals and objectives of programs designed to serve such students. Second, 
instruments not published and commercially distributed are typically not included in the 
collections. Thus, locally developed instruments used in portfolio reviews, audition ratings, 
evaluation of student products, etc. would not be included, leaving a major gap in 
information which provides advice on the availability and quality of such instruments. 

Listings and reviews of identification instruments such as those provided by the 
study at the Educational Improvement Center and various textbooks also have relied largely 
on established tests or publishers to provide information on available tests. These sources 
generally limited their categorization to the traditional listings of types of giftedness 
(intellectual, specific academic ability, creativity, leadership, and visual and performing arts) 
and do not evaluate of their effectiveness for identifying specific attributes or use within 
specific populations. 

The National Repository for Instruments and Strategies Used in the Identification 
and Evaluation of Gifted Programs is distinctive in several ways. Reviews of instruments 
focus on the reliability and validity of the instruments as they are used specifically in 
programs for the gifted; that is, the focus is on specific constructs of giftedness used to 
identify gifted students (identification) or on specific outcomes or goals set for gifted 
programs (evaluation). Further, the reviews examine the degree to which specific attributes 
are measured within those contexts and the degree to which the instruments are useful in 
identifying specific sub-populations of gifted students when used with specific systems of 
identification. 
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Report Overview 

Because different portions of the project had different methodologies, each chapter 
of this report centers on one aspect of the study. This chapter presents the establishment of 
the National Repository. Chapter 2 presents the review of current literature and 
identification practices based on documents submitted to the National Repository. Chapter 
3 includes information on the reliability and validity of selected locally developed 
identification instruments. The findings of the evaluation utilization study and the review of 
current evaluation literature and practices are presented in a separate monograph entitled 
Instruments and Evaluation Designs Used in Gifted Programs. 



Establishment of the National Repository 

Methodology 



Mailings 

To gather as many instruments, identification strategies, and evaluation designs as 
possible, we designed a process to gather information on both standardized and locally 
developed instruments, and to identify state and local evaluation designs and systems for 
identification. Specific efforts were made to identify instruments and strategies which had 
been used in the identification of minority, economically disadvantaged, non-English 
speaking, and handicapped gifted students and in evaluating programs for these students. 

Four strategies were employed to collect the instruments, systems, and designs 
which have been used for program evaluation and student identification at the national, state, 
regional, and local levels. First, a letter requesting all state criteria used in identification 
systems, state recommended identification instruments, state-wide evaluation reports, and 
evaluation instruments was sent to each official in the state departments of education who 
had been designated (as of Fall, 1990) as having responsibility for gifted and talented 
programs. These individuals were asked to supply copies of any identification or evaluation 
instruments being used on a state, regional, and/or local level, or to provide a list of district 
level personnel who could be contacted for such information. They were asked to furnish 
the name of the developer of the instrument, information on how the instrument was used, 
who used it (e.g., psychologist, teacher, evaluator), and how data were analyzed. State 
officials were advised that they could submit state guidelines, evaluation reports or other 
documents from which we would glean the necessary information if that were more 
convenient. 

Next, Collaborative School Districts (a CSD is a school district that specifically 
agreed to work on NRC/GT projects) were asked to provide any instruments used in 
identifying gifted students, a description of identification procedures used, demographic 
information on students selected, and copies of any evaluations of their programs or 
projects in gifted education. They were also asked the name of the instrument developer 
and the uses of the instrument (e.g., identification, evaluation). We also asked that, 
whenever possible, the name of the evaluator be provided with evaluation reports. 

A similar letter and form were sent to approximately 5,000 school districts across 
the United States. Addresses for these districts were obtained from an educational database 
firm. Where possible, we delivered the letters at state conferences (Florida, Iowa, and 
Virginia), through state association mailings (Texas), or through state gifted coordinators 
(Colorado and Arizona). 
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We recognized, of course, that districts might not be comfortable with their current 
identification procedures or instruments, or districts might realize that they didn't truly abide 
by stated procedures or state regulations, and therefore, might be reluctant to respond 
accurately (or at all) to the survey. We attempted to avoid bias that might arise in the 
responses in two ways. First, districts were assured that information would be strictly 
confidential and we would not reveal names of districts in our reporting of data without the 
school district's permission. Second, our survey clearly emphasized that we were interested 
in all data about instruments and surveys, including instruments or systems which didn't 
seem to work as intended. We stressed the importance of learning from the things that do 
not function as expected, as well as learning from the things that do work. Requests 
concerning the value of each instrument also sought respondent information on the positive 
and negative aspects of the instruments in general as well as information on identifying 
students from specific underserved populations. Finally, a random sample of non- 
respondents was contacted by follow-up letter to determine whether there had been a 
systematic response bias. 

All contacts were asked specifically to indicate instruments, strategies, and data 
sources which they believed had been particularly useful in identifying minority, 
economically disadvantaged, underachieving, non-English speaking, and/or handicapped 
gifted students. The Council for Exceptional Children and state department personnel were 
asked for lists of institutions which specifically serve individuals who are blind, hearing 
impaired, or with other handicapping conditions. These institutions were contacted 
specifically and directly. In addition, all individuals contacted were asked for program 
evaluation instruments, including process and product/performance ratings as well as 
standardized tests. 

Announcements 

Professional organizations, journals and state associations through which it would 
be appropriate to make requests for information were identified, and specifically tailored 
announcements and letters were sent to each association and journal. In addition, 
announcements were included in the conference programs and/or registration packets at the 
annual meetings of the National Association for Gifted Children and the American 
Evaluation Association. 

Responses 

The mailings and announcements yielded responses containing identification 
information from 542 individual school districts. An additional 65 school districts 
responded that they would have liked to forward materials, but could not do so because the 
program had recently been cut or was undergoing extensive changes. A random sample of 
140 non-responding CSDs and 100 additional non-responding local education agencies 
(LEAs), but not CSDs was sent a questionnaire asking why they had not responded. Of 
these, 45 CSDs and 44 LEAs returned the questionnaire. 

Table 1 summarizes the reasons they provided for not responding. The number of 
respondents indicating that they do not have programs is of interest because the names of 
contacts in school districts were obtained from the database only if they were listed as a 
coordinator of gifted programs. Because many of these districts indicated that the program 
had been terminated only recently, we were obviously limited in our sampling by current 
program instability. The responses indicating limited resources, transitional, inadequate, and 
new programs might indicate a dissatisfaction with programs as they currently exist and an 
accompanying effort to change them, restrictions on resources due to the current economic 
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situations in schools, or changes resulting from the heterogeneous grouping movement 
within public education. 

New file materials continue to arrive and are added to the data bank as they are 
received. Follow-up letters seeking additional information have been necessary for nearly 
every district. 

Reviews of the Literature 

Database searches were conducted across Educational Resources Information 
Center (ERIC), PsycLIT (the computerized version of Psychological Index), Dissertation 
Abstracts International, and VIRGO (the University of Virginia computerized card 
catalogue system). Search terms included gifted, ratings, scales, reliability, validity, tests, 
measurements, identification, evaluation, and utilization, and these terms were used singly or 
in combination as appropriate. Each search yielded a list of potential resources which were 
reviewed for information on the state of the art in identification or evaluation (particularly 
evaluation utilization), information on use of particular instruments or strategies for 
identification or evaluation, and information on reliability and validity. 

The initial search yielded 375 documents including approximately 174 journal 
articles, 16 books, 37 dissertations, and 120 ERIC documents. In some cases dissertations 
were obtained directly from the authors. Large ERIC documents were reviewed on 
microfiche with copies made of relevant sections only. Abstracts of each document were 
prepared by staff focusing particularly on either test review information or usefulness in 
identifying underserved gifted students. 



Table 1 

Content Analysis of Reasons for not Submitting Information to the Databases ( N = 89) 



Reason Given 


n 


% 




Non-existent Program 


29 


33 




Resource Limitations 


19 


21 




Transitional Program 


16 


18 




Non-unique Methods 


15 


17 




Inadequate Program 


9 


10 




New Program 


6 


7 




No Program Evaluation 


6 


7 




Just Don't Want to 


6 


7 




Working on Another Research Project 


5 


6 




Lost/Didn't Receive Request 


4 


5 




Other 


5 


6 





Note. Respondents sometimes indicated more than one reason for not responding. 

O 
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The information compiled from the resources listed above yielded seven databases 
on identification as part of the National Repository. The computer databases cover three 
categories of information: bibliographic entries, standardized instrument reviews and use, 
and locally developed materials. The bibliographic databases contain abstracts of published 
reviews of standardized instruments, abstracts of articles about the use of standardized 
instruments in identification, and abstracts of articles about particular issues in identification 
(e.g., underserved populations). The standardized instrument databases include listings of 
the ways in which published instruments are used (see Categorizing and Cataloging 
Identification Instruments below) and reviews of the instruments on The Scale for 
Evaluating Gifted Identification Instruments ( SEGI1 ), a scale developed by NRC/GT staff. 
The local instrument databases include listings of a collection of identification instruments 
developed and used at the local school level but not published. Within each database, the 
entries are further divided into two groups — those we have permission to share with the 
public and those we do not. A complete list of the identification database names, content 
descriptions, and number of entries appears as Table 2. Full descriptions of the ways in 
which entries were made in databases and fuller description of the databases are available 
upon request. The particular categories were created in order to facilitate searches for 
information by project staff and ultimately by educators, psychologists, and parents seeking 
information from the databases. While a particular article might relate to more than one 
category, it was classified by the dominant theme of the article. 

A mailing was sent to all contributors of locally developed instruments asking for 
permission to release these materials. Only materials from school districts that have given 
permission for distribution are included in the database used to fill requests for local 
instruments, although all instruments were included when analyses of the data were 
conducted for our report. Any local instrument released also contains the name and address 
of a contact person in the district which developed the instrument. A copy of the order form 
used to obtain database information is included in Appendix A. 



Categorizing and Cataloging Identification Instruments 

Establishing Database Categories 

A coding guide for recording individual school definitions of giftedness, types of 
instruments used, respondents, etc. was developed by project staff. Definitional categories 
were based on a review of current textbooks and journal articles on giftedness and 
intelligence. Types of instruments, respondents, gifted constructs, etc. began as a list of 
traditional categories from the measurement and gifted fields and were revised as 
instruments were reviewed. Several categories were sub-divided (e.g., creativity was split 
into creativity — ideation and creativity — problem solving as it became clear that both 
instruments and current theory differentiated between these constructs. A complete listing 
of categories is found in Figure 1. This guide allowed us to categorize definitions of 
giftedness used in the identification process (e.g., United States Office of Education 
definition [USOE], Renzulli's Three-Ring Definition, Structure of the Intellect definition), 
the respondent, the type of instrument (checklist, standardized instrument), the underlying 
gifted construct, the grade level at which the instrument was used. Frequency counts of 
definition, constructs, published instruments, and underserved populations were conducted, 
and are reported in Chapter 3 of this document. 
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Table 2 

Summary of Databases on the Identification of Gifted Students 



Database Name 


Description of Contents 


Number of 
Entries* 


IDREVIEW 


published reviews of identification instruments 


201 


IDISSUES 


articles and papers which address specific issues in 
identification of gifted students (e.g., underserved 
populations) 


152 


ID ARTICLES 


articles and papers which address use(s) of particular 
identification instruments (e.g., using the WISC-R in 
low SES populations) 


149 


EDLOCAL 


identification instruments developed and used by local 
school districts 


1,762 


SEGII 


reviews of published identification instruments on the 
standard NRC/GT scale 


350 


PUB 


records of the use of standardized identification 
instruments by local school districts 


217 


IDSTATE 


contents of state department of education documents 


26 



* As of 3/1/93. 



Parallel procedures were used to code and enter information on articles, books, test 
reviews, chapters, and other information contained in the databases. These procedures are 
documented in The Revised Ultimate National Research Center on the Gifted and 
Talented Guidebook (Version 4) Files and Databases of the National Repository of 
Identification and Evaluation Instruments which is available upon request. An abridged 
version of a sample database output based on a request for articles on identification of the 
African-American students in the category of general intellectual ability is included in 
Appendix B. 

Assessing the Psychometric Properties of Published Instruments 

The second line of investigation focused on reviewing published instruments which 
were either cited in journal articles reviewed or identified by school districts. This phase 
was subdivided into two parts. 

Initially, the staff gathered all available data from the printed literature and from the 
school division survey responses on the reliability, validity, examinee appropriateness, 
norms, usability, teaching feedback, and ethical propriety of the instruments. 
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Category: General Definition 
General IQ 

United States Office of Education (USOE) 
Three-Ring 

Information Processing 
Multiple Intelligences 
Tannenbaum 
Structure of the Intellect 
Triarchic Theory 
Other 

Not Available/Applicable 



Category: Gifted Construct 

General Intellectual Ability 

Specific Intellectual Abilities 

General Academic Ability 

Verbal/Linguistic Ability 

Mathematical/Logical Ability 

Scientific Aptitude 

Sculpting Ability 

Photography Ability 

Other Visui Arts Ability 

Music Performance Ability — Voice 

Music Performance Ability — Instrumental 

Music Composition Ability 

Dance Ability 

Acting Ability 

Painting/Drawing Ability 

Other Performing Arts Ability 

Vocational Education/Practical Arts Ability 

Inter/Intra-personal Ability/Leadership/Psycho-Social Ability 

Creativity: Ideation 

Creativity: Problem Solving 

Task Commitment/Motivation 

Psychomotor/Bodily-Kinesthetic Ability 



Category: Ethnic/Minority Considerations 

African-American 
Hispanic-American 
Asian-American 
Native American 
Polynesian 

Racial/Ethnic Minorities — General 
Other Ethnic/Minority Groups 



(figure continues ) 

Figure 1. Published instruments: Categories and specific classification scheme. 



O 

ERIC 
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Category: Language Considerations 

Limited English Speaking 

Category: Population Considerations 

Urban 

Suburban 

Rural 



Category: Socio-economic Considerations 
Low SES 

Category: General Considerations 

Female 

Male 



Category: Handicapped Considerations 

Learning Disabled 
Hearing Impaired 
Visually Impaired 
Physically Challenged 
General Handicapped 



Category: Underachievement 



( continued) 

Figure 1. Published instruments: Categories and specific classification scheme. 



These technical data were used to rate each published instrument using a model 
rating scale developed by project staff, but based on earlier work done by the Evaluation 
Technologies Program of the Center for the Study of Education and the Humanizing 
Learning Program of Research for Better Schools, Inc. in their series of test evaluations 
(Hoepfner et al., 1976; Hoepfner et al., 1972; Hoepfner, Stem, & Nummedal, 1971). The 
existing rating scale was modified to reflect the specific uses to which these instruments 
have been put — identification using a specific construct or definition of giftedness. The 
measurement standards of the Standards for Evaluations of Educational Programs, 
Projects and Materials (Joint Committee on Standards for Educational Evaluation, 1981), 
the Standards for Educational and Psychological Testing (American Educational Research 
Association, American Psychological Association, and National Council on Measurement in 
Education, 1985), and Guidelines for Test Use (Brown, 1980) were used in developing the 
final tool for assessing the instruments. This tool, the Scale for Evaluating Gifted 
Identification Instruments (SEGII), and relevant technical data are described in full detail in 
Callahan, Lundberg, & Hunsaker, (1993). The technical manual for the Scale for the 
Evaluation of Gifted Identification Instruments is found in Appendix C; a copy of the 
instrument is found in Appendix D; and interrater agreement percentages are included in 
Appendix E. 
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For each published instrument listed in the Repository, we identified each definition 
and construct which school districts named as the focus in the use of that instrument. For 
each construct named, the instrument was reviewed with that construct as a focus of the 
review. Hence, any particular instrument might be rated once, twice, or several times. A 
total of 78 tests have been reviewed. Appendix F contains a complete listing of the reviews 
by construct and instrument. 

Local instruments were also recorded by definition and construct of giftedness 
assessed, by type of instrument, population assessed, etc. Although many instruments were 
provided to the NRC/GT, only one school district provided any information on the 
reliability and validity of the instrument. Our guidelines for rating instruments were based 
on the judgement that instruments lacking evidence of reliability and validity could not be 
recommended, and hence, would not be reviewed further. 

The staff of the project also reviewed entries from ERIC, journal articles, and other 
sources on tests. All of these data on published instrument reviews, which can be accessed in 
the IDREVIEW and ID ARTICLE databases, have been condensed. 

Importance of This Repository 

Appropriate program development and modification is based on the collection of 
valid and useful data on the functioning of a program. Administrators of programs for the 
gifted have lacked access to instruments which have been validated or even demonstrated to 
be reliable for measuring most components of their programs. The collection of 
instruments in a central repository and an evaluation of these instruments by individuals 
with expertise in evaluation, psychometrics, and gifted education is long overdue in the field 
of gifted education. Many districts have struggled with the search for such instruments; 
some have made initial development efforts; some have collected some data on the 
effectiveness of instruments. The National Repository information provides more general 
access to a wider range of information by school district personnel. 

The purpose of the databases is to allow practitioners to summon information on 
instruments school districts are using to assess various constructs of giftedness, to access 
information on the qualities of particular instruments, or to allow access to what other 
districts are doing to identify particular underserved groups of gifted students. An abridged 
sample of response to a request for a search is presented in Appendix B. 
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CHAPTER 2: The Literature on Recommended Practices in 
Identification and the Current Practice 



A review of the practices recommended in the literature by experts on gifted 
education was used as a basis for making comparisons between "ideal" practices and the 
"actual" practices which were reflected in the information on identification practices and 
instruments submitted by school districts to the National Repository. 

As conceptions of intelligence and ability have evolved, expanded definitions of 
giftedness have come to gain wider acceptance. Experts in gifted education accordingly 
have provided recommendations so that assessment practices in the identification of gifted 
children could keep pace. The literature review which follows focuses on recommendations 
made over the last 15 years. These recommended practices appeared consistently in the 
literature, with rare refiitation. Specific references supporting the recommendations are cited 
within each section. 



Broadened Conceptions of Giftedness 

To ensure that gifted programs better serve their population, it is important for 
educators to select a well-defined concept of giftedness and to use that concept to determine 
identification procedures (Kontos, Carter, Ormrod, & Cooney, 1983). Currently, many 
definitions of giftedness exist. Nearly all current conceptions of intellectual functioning and 
giftedness go beyond a narrow concept of intellectual ability as a unitary trait measured by 
traditional intelligence tests. For example, Sternberg (1986) identified analytic, synthetic, 
and practical intelligences as distinct areas of cognitive functioning in which individuals may 
excel; Gardner (1983) posited seven intelligences (linguistic, logical, mathematical, spatial, 
musical, interpersonal, intrapersonal, and bodily kinesthetic). 

Not only has the conception of intelligence expanded, so has the conception of 
giftedness. Once viewed as an exclusive domain of the "intellect" and traditionally 
associated with excelling across many school-related domains, current definitions have 
explored extending to domains outside of school (e.g., Renzulli's Three-Ring definition — 
above average ability, creativity, and task commitment, 1988) and outside of die traditional 
academics (e.g., the USOE definition — general intellectual ability, specific academic ability, 
creative or productive thinking, leadership ability, visual and performing arts, and 
psychomotor ability, Marland, 1972) and to looking for specific abilities (e.g., mathematical 
talent, Stanley & Benbow, 1986). 

Some current definitions include nonintellective as well as cognitive factors. For 
example, Renzulli (1988) considers above average academic ability, creativity, and task 
commitment and Tannenbaum (1986) incorporates superior general intellect and personal 
environmental factors into his definition. The traits within each definition overlap to denote 
"giftedness." Gardner's Multiple Intelligence Theory (1983) also includes nonintellective 
factors, such as interpersonal skills and intrapersonal knowledge; however, in Gardner's 
framework, these factors are seen as being expressed as independent domains. 
Developments in the study of cognition and the expansion of conceptions of giftedness 
clearly suggest that school personnel need to re-think identification procedures which are 
based primarily on intellectual assessment using only traditional measures of intelligence to 
see whether such identification measures are truly consistent with the gifted populations 
they intend to serve. 
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Using Multiple Criteria 

The recommendation that multiple criteria be used in the identification of the gifted 
is pervasive in the literature (Ebmeier & Schmulbach, 1989; Frasier, 1987; Kirschenbaum, 
1983; Roach & Bell, 1986). Use of multiple criteria in the decision-making process is 
characterized by the use of both standardized and non-standardized instruments, process 
and performance indicators, and multiple sources of data (e.g., student, teacher, parent, or 
peers). This recommendation has found its way into the policy statements of 32 states 
(Houseman, 1987). Several studies have reported success in the use of multiple criteria 
(Dirks & Quarfoth, 1981; Ehrlich, 1986). A single measure/single criterion is still used in 
many instances for selecting students for special programs for the gifted (e.g., Hoge, 1989). 

Unfortunately, the use of multiple criteria can be misinterpreted as "multiple 
hurdles." The intent of the use of multiple criteria is to give professionals the most 
complete picture of the student and to allow many ways for a student to exhibit talent. Its 
intention is not to create a situation in which students must meet all criteria in order to be 
considered "gifted." To ensure more reasonable application, this recommendation might be 
better phrased as a recommendation to use alternative pathways to identification with 
multiple sources of data used as part of the decision-making process. 



Unique Instrumentation for Different Areas of Giftedness 

The literature (e.g., Culross, 1989; Frasier, 1987; Goldberg, 1986; Johnsen, 1986) 
supports defining and serving talents across a diverse spectrum. Experts recommend that 
school district personnel seek out those identification strategies appropriate for the specific 
domains the school district has elected to serve (Gallagher, 1985; Platow, 1984; Renzulli, 
Reis, & Smith, 1981). That is, if a school district purports to measure general intellectual 
ability, specific academic ability or achievement, music, and dance, then there should be 
separate instruments and procedures should be considered for each of these areas. 

Though this recommendation is logical, its implementation is hampered by slow 
development in the assessment field. Only general intellectual ability and some academic 
talent areas have been well researched and defined, resulting in related standardized test 
instruments having evidence of acceptable levels of reliability and validity. Other commonly 
cited areas of giftedness, such as leadership, creative or productive thinking, and abilities in 
the arts remain less clearly defined and measurement of their parameters relatively 
unspecified (Goldberg, 1986; Johnsen, 1986). Alternative assessment tools and strategies 
for assessing independent domains of giftedness have been discussed in the literature for 
many years. As early as 1981, Treffinger, Renzulli, and Feldhusen recommended including 
creativity profiles and products, and Khatena in 1982 suggested auditions to assess visual 
and performing arts. 

Despite the arguments that assessments of less traditional aspects of giftedness do 
not lend themselves readily to standardization and quantification (Goldberg, 1986) and that 
assessments generally depend on some form of pooled judgments (Khatena, 1982), it is still 
imperative that any test, rating scale or other assessment tool have suitable evidence of 
reliability and validity for assessing the area of talent being considered. If there are 
differences in the underlying constructs of the ability necessary to create artistic products 
and scientific breakthroughs, the identification of talent in these domains must be distinct 
with both instrument selection and identification procedures based on the underlying 
concept of talent in that domain. There are no "one-size fits all" instruments. 
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Reliability and Validity of Construct 

If no theoretically based definition of giftedness will fit all programs and 
circumstances (Roach & Bell, 1986), the criteria for the selection of gifted students must 
match the original rationale or definition of giftedness held by the school district (Hoge, 
1989; Williams, 1988). The credibility of the identification decision-making process rests 
on the instruments and strategies that allow for reliable and valid measurement relative to the 
construct under consideration. 

Most programs for the gifted continue to focus on general intellectual ability and 
specific academic areas (Johnsen, 1986). While tests of intelligence provide relatively 
objective, reliable, and valid measures of general intellectual ability in the sense of predicting 
general school achievement, they give little information about specific talents, even in the 
intellectual domain (Goldberg, 1986). Assessment using only traditional intelligence tests 
appears questionable for assessing across all the constructs within the broadened 
conception of giftedness and inappropriate for use in identifying ability in specific academic 
areas, the arts, creativity, or leadership. 

Questions have been raised about the validity of instruments used for identification 
of giftedness across all definitions and components of giftedness, including the USOE 
definition. Studies examining the viability of assessing gifted children in the area of general 
intellectual ability, for example, have produced varying findings as to whether the 
instruments had a strong relationship to the construct being assessed (Carvajal & McKnab, 
1990; Mather & Udall, 1985; O'Tuel, Ward, & Rawl, 1983). In another domain, that of 
creativity, Runco (1986) found that the assessments of the various constructs lack 
discriminate validity. Predictive validity related to gifted program performance has not been 
established for most instruments used in the identification of gifted students (O'Tuel, Ward, 
& Rawl, 1983). Further, intelligence, achievement, and creativity tests have failed in 
establishing predictive validity for adult success within the gifted population 
(Kirschenbaum, 1983). It behooves educators at all levels to examine the predictive validity 
of instruments used to identify gifted students. 



Limitations of a Single Score Cut-Off Determination of Giftedness 

The use of an arbitrarily rigid cut-off IQ score or summed matrix score as the basis 
of identification has been widely criticized in the literature (Chang, 1985; Culross, 1989; 
Ebmeier & Schmulbach, 1989; Frasier, 1987; Goldberg, 1986; Johnsen, 1986; 
Kirschenbaum, 1983; Renzulli & Delcourt, 1986; Treffinger, 1982c). Further, the score of 
an individual on a single instrument is best conceived as a range of scores, not a single 
point, due to errors in measurement (Kirschenbaum, 1983). Yet, many districts have used 
cutoff scores as the sole basis of identification either independently or because of the 
guidelines or policies of the state (Treffinger, 1982c). In some states, for example, to be 
eligible for a gifted program, the sole criterion a student must achieve is a minimum score 
on an intelligence test (Goldberg, 1986; Houseman, 1987). In some school districts, group 
standardized achievement test scores may be used as a cutoff in determining who enters the 
pool (Johnsen, 1986) for further screening. 

While intelligence test and standardized achievement test scores are relatively stable 
and consistent scores, the use of these scores rigidly and alone as a criterion for identifying 
gifted students belies current theory that giftedness includes non-intellective factors, that 
giftedness may manifest itself through a variety of means of expression, and that giftedness 
may be domain specific. Intelligence tests are best regarded as reliable indicators of analytic 
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skills which predict school achievement very accurately. They are useful as part of a full 
process of screening and identifying giftedness when giftedness is defined as a global 
construct predicting school achievement. Similarly, the use of a single achievement test 
score is flawed. 

The use of intelligence and/or achievement test scores is appropriate when (a) the 
definition of giftedness matches the construct measured by the instrument, (b) the score is 
viewed as a band of scores incorporating the standard error of measurement of the test, and 
(c) the score is part of a full consideration of both cognitive and non-intellective factors 
contributing to giftedness. 



Identification and Placement Based on Student Need 

In some cases, the number of students accepted as gifted and talented has been 
determined by the number who can be accommodated by a particular school, class, or 
program (Goldberg, 1986; Johnsen, 1986). Very often, financial exigencies dictate the 
definition of giftedness and the criteria used by state and local school systems. Applying a 
percentage quota system means abandoning conceptual program standards and selection 
criteria, and jeopardizes the concept of fair selection (Fetterman, 1986). Treffinger (1982b) 
cautions against singling out a prespecified percentage of students. Instead, we should be 
concerned with defining and screening for the unique characteristics associated with our 
concept of giftedness. 



Instruments for Underserved Populations 

Special attention should be given to the different ways in which children from 
different cultures manifest behavioral aspects of giftedness (Frasier, 1987). No one set of 
standards can be applied fairly to all types of gifted students (Goldberg, 1986). Although 
efforts have been made to include children from all ethnic and racial minority groups in 
gifted programs, some claim that minority groups are still grossly underrepresented by 
approximately 30% to 70% (Richert, 1985a; Sapon-Shevin, 1987). In the NELS 88 sample 
(National Educational Longitudinal Study), minorities as a whole were not 
underrepresented, but Hispanic students were somewhat underrepresented and American 
Indian/Alaskan Native students were severely underrepresented. However, the participation 
rate of students from the lowest socioeconomic status (lowest 25% of the population) was 
only 3.2% (O'Connell-Ross, 1992). 

A frequent explanation for low representation of children of poverty or children 
representing minority groups in gifted programs is performance below the cutoff score on 
standardized achievement or IQ tests which have led several experts to conclude that such 
tests may be culturally biased (Alvino, McDonnel, & Richert, 1981; Deschamp & Robson, 
1984; Frasier, 1989; Harrington, 1982; Johnsen, 1986; Lundberg & Callahan, 1992; 
McKenzie, 1986). Many experts have concluded that nontraditional methods are more 
effective than traditional instruments for testing the underserved gifted student because the 
instruments typically used to assess intelligence or academic performance do not measure 
the different ways in which children of differing backgrounds may manifest gifted 
behaviors (Frasier, 1987; Masten, 1985; Ryan, 1983; Sisk, 1988). Unfortunately, the same 
instruments are used with and standards are applied to, the dominant, middle-class 
population as are applied to diverse, underserved populations (Goldberg, 1986). 
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CHAPTER 3: Analysis of the Database Entries 



An important study on the use of identification instruments was conducted over ten 
years ago (Alvino, McDonnel, & Richert, 1981). Subsequently, Richert (1985b) reported 
on the aspect of that project which analyzed published tests in use at that time to identify 
gifted and talented youth in the United States. With the USOE definition as a basis for 
reviewing these tests, they found: (a) a misunderstanding or lack of application of the 
federal definition in its broadest sense; (b) uses of instruments to assess constructs not 
originally intended and for populations or abilities for which no validity evidence existed; 
(c) biased tests and procedures that screened out disadvantaged students; (d) multiple 
criteria combined in statistically unsound ways; and (e) instruments and procedures used at 
inappropriate stages of identification. 

In view of the issues raised by Richert and the recommendations for improved 
practice given since that time, we set out to determine how published instruments currently 
are used in identifying gifted students. 



Method 

As part of the activity of The National Research Center on the Gifted and Talented 
(NRC/GT), researchers at the University of Virginia site conducted a comprehensive study 
of instruments used in gifted programs throughout the United States. To briefly review 
information from Chapter 1: Specific requests for information about instruments used in 
the identification of gifted students and the evaluation of gifted programs were sent to all the 
Collaborative School Districts associated with the NRC/GT. Mailings were sent to about 
5,000 gifted program coordinators around the country based on a marketing mailing list. In 
addition, we searched databases such as ERIC, PsycLIT, and Dissertation Abstracts 
International for descriptions of local gifted programs. Finally, appeals were made to over 
fifty professional organizations such as the National Association for Gifted Children and 
the Association of Supervision and Curriculum Development through their journals and at 
their conventions. From these efforts, over 1,200 files were created, each focusing on 
materials from the responding district. When information was not complete, a follow-up 
letter was sent to the school district asking for clarification. The present study is based on 
information from the 551 files which were complete enough to allow comparison of specific 
instruments to definition. 



Data Analysis 

A classification system for recording individual components of a school's gifted 
identification system was developed based on survey texts in gifted education (e.g., Clark, 
1988; Colangelo & Davis, 1991; Davis & Rimm, 1984; Tannenbaum, 1983). School 
documents were reviewed to identify definitions of giftedness. Then the journal articles 
collected in the search were reviewed and only new definitions were added. Similarly, we 
looked for listings of specific constructs associated with the definitions. For example, 
Renzulli’s Three-Ring definition includes the constructs of above average ability in a 
specific domain; the USOE definition includes general intellectual ability, specific academic 
ability, creative or productive thinking, leadership ability, visual and performing arts, and 
psychomotor ability. Categories reflecting specific issues in identification were also created 
based on the literature review. For example, the low incidence of female participation in 
secondary programs in mathematics (e.g., SMPY) led us to look for special instruments or 
strategies that might be used to identify female talent in that domain. Information from each 
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school district was recorded according to these categories. New categories and codes were 
created to better reflect the uses made of the instruments when a particular instrument or 
strategy did not fit the original classification scheme. The major categories and the specific 
classifications under each category are detailed in Figure 1 (p. 8). 

Coding of definitions, constructs, and tests was completed by a staff member of the 
NRC/GT at The University of Virginia. A random sample of the codings was reviewed by 
other staff to clarify constructs and to verify accuracy. When differences in understanding 
were discovered, previously coded information was recoded. Ambiguous items were 
referred to the project staff for clarification and consensus in coding. 

The first coding decision was to determine the definition of giftedness used by the 
school district and the constructs adopted under that definition. For example, if a school 
district used the USOE definition, we then determined which of the specific areas of 
giftedness it chose to serve. The next step was to match the instruments the school districts 
reported using with the construct of giftedness they had indicated in their definitions or 
other program documents. Finally, instruments used to provide additional information on 
special populations such as ethnic minorities or the disabled were coded. Only instruments 
available from a publishing house and with specific instructions for administration, scoring, 
and interpretation were considered "standardized published" assessments. Other 
instruments were not considered for the analysis reported in this section. 



Results 

Definitions 

Table 3 illustrates the distribution of adoption of particular definitions of gifted and 
talented. The USOE definition was, by far, the most widely accepted definition of 
giftedness (48%), with IQ (11%), and Three-Ring definition (8%) as the other two widely 
utilized definitions in our respondent group. Nine districts adopted both the Three-Ring 
and USOE definitions. One district cited the Multiple Intelligence definition and one 
district reported basing its definition on the Structure of Intellect model of intelligence. No 
districts reported adopting the Information Processing, Triarchic, or Tannenbaum 
definitions which is probably not surprising given that these definitions are not frequently 
recommended in state guidelines (Houseman, 1987). Eighteen districts provided definitions 
not derived directly from the literature. For example, one district chose to use high 
academic achievement as its sole definition for giftedness. Four districts stated they used 
no definition of giftedness and 165 did not report a definition. Follow-up inquiries to these 
districts resulted in no further clarification. Several responded that they had provided all 
information available. 
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